Update spack-stack-dev from spack develop as of 2024/10/31 #482

climbfuji · 2024-11-01T00:27:57Z

Description

Update spack-stack-dev from spack develop as of 2024/10/31. The following files had merge conflicts that I needed to resolve and that I would like the reviewers to look at (there were a few more, but those I knew enough about to resolve with confidence):

        both modified:   .github/workflows/unit_tests.yaml
        both modified:   var/spack/repos/builtin/packages/fontconfig/package.py
        both modified:   var/spack/repos/builtin/packages/g2/package.py
        both modified:   var/spack/repos/builtin/packages/gdal/package.py
        both modified:   var/spack/repos/builtin/packages/gmake/package.py
        both modified:   var/spack/repos/builtin/packages/harfbuzz/package.py
        both modified:   var/spack/repos/builtin/packages/jasper/package.py
        both modified:   var/spack/repos/builtin/packages/landsfcutil/package.py
        both modified:   var/spack/repos/builtin/packages/libtirpc/package.py
        both modified:   var/spack/repos/builtin/packages/libxkbcommon/package.py
        both modified:   var/spack/repos/builtin/packages/openmpi/package.py
        both modified:   var/spack/repos/builtin/packages/pixman/package.py
        both modified:   var/spack/repos/builtin/packages/py-cffi/package.py
        both modified:   var/spack/repos/builtin/packages/py-metpy/package.py
        both modified:   var/spack/repos/builtin/packages/py-setuptools-rust/package.py
        both modified:   var/spack/repos/builtin/packages/qt/package.py
        both modified:   var/spack/repos/builtin/packages/sfcio/package.py
        both modified:   var/spack/repos/builtin/packages/sigio/package.py
        both modified:   var/spack/repos/builtin/packages/ufs-weather-model/package.py

Update 2024/11/08 Added commit a3dd052 to fix a bug in cprnc - the install rpaths for the netCDF libraries were missing, which caused the installations from the binary cache to fail during the relocation step:

$ ldd install/bin/cprnc
        linux-vdso.so.1 (0x00007ffde9de5000)
        libnetcdff.so.7 => not found
        libnetcdf.so.19 => not found
        libgfortran.so.5 => /home/dom/work/spack-stack/update_from_spack_dev_20241031/spack-stack-update/envs/ue-gcc-13.3.0/install/gcc/13.3.0/gcc-runtime-13.3.0-3gtnokv/lib/libgfortran.so.5 (0x0000152b8aa7d000)
        libm.so.6 => /lib64/libm.so.6 (0x0000152b8a9a2000)
        libgcc_s.so.1 => /home/dom/work/spack-stack/update_from_spack_dev_20241031/spack-stack-update/envs/ue-gcc-13.3.0/install/gcc/13.3.0/gcc-runtime-13.3.0-3gtnokv/lib/libgcc_s.so.1 (0x0000152b8a97c000)
        libc.so.6 => /lib64/libc.so.6 (0x0000152b8a773000)
        /lib64/ld-linux-x86-64.so.2 (0x0000152b8ad63000)

Testing

Unit tests (GitHub actions) - note that I had to turn off the Windows Github actions tests, since they are written in a way that require the GitHub repo to be https://github.com/spack/spack
For testing with spack-stack, see Update spack-stack-dev from spack develop as of 2024/10/31 spack-stack#1363 (work in progress)

…nherited cmake (spack#47022)

Co-authored-by: Gava, Francesco <[email protected]> Co-authored-by: Bernhard Kaindl <[email protected]>

* e4s external rocm ci: upgrade to v6.2.1 * use ghcr.io/spack/spack/ubuntu22.04-runner-amd64-gcc-11.4-rocm6.2.1:2024.10.08 * magma +rocm: add entry for v6.2.1

Co-authored-by: LMS Bioinformatics <[email protected]>

* Updates Caliper recipe to build the new Python bindings * Implements setup_run_environment for Caliper to update PYTHONPATH

…k#47029) and add another backward compat bound for just 8.8

* e4s ci stacks: add fftx cpu, cuda, and rocm builds * disable fftx+rocm due to spack github issue spack#47034 * e4s oneapi: fftx has spack build error spack#47048

Co-authored-by: Bernhard Kaindl <[email protected]>

* voropp: migrate to cmake * lammps: update voropp dep

* Remove the implicit CORE-AVX512 since the CPU specific flags are added by the compiler wrappers. * Add `-i_use-path` to help `ifx` find `lld` even if `-gcc-name` is set in `ifx.cfg`. This file is written by `intel-oneapi-compilers` package to find the correct `gcc`. Not being able to find `ldd` is a bug in `ifx`. @rschon2 found this workaround.

…k#47023)

…#47065) Turns out `os=...` of the spec and `MACOSX_DEPLOYMENT_TARGET` are kept in sync, and the env variable is used to initialize `CMAKE_MACOSX_DEPLOYMENT_TARGET`. In bootstrap code we set the env variable, so these bits are redundant. --------- Co-authored-by: haampie <[email protected]>

…spack#47077)

Some unused methods in VTK-m resulted in compile errors. These were not discovered because many compilers ignore unused methods in templated classes, but the SYCL compiler for Aurora gave an error.

Co-authored-by: Peter Scheibel <[email protected]>

Co-authored-by: Alex C Leute <[email protected]>

* py-coca-pytorch: new package * [py-coca-pytorch] black --------- Co-authored-by: Alex C Leute <[email protected]>

Co-authored-by: Alex C Leute <[email protected]>

…ge.py, var/spack/repos/builtin/packages/libxkbcommon/package.py, var/spack/repos/builtin/packages/openmpi/package.py

…and lib/spack/spack/compilers/oneapi.py: set correct version ranges

AlexanderRichert-NOAA · 2024-11-07T18:01:19Z

I'll need a bit of time to look over this. It looks good so far, though I think it'll need one or two tweaks related to static libraries...

srherbener

Tested on my Mac (Sonoma 14.6.1, Command Line Tools 15.3, GNU Fortran 12.4.0), and it looks good. I was able to build jedi-bundle and I got the usual ctest failures (19) that we have been seeing on the mac.

climbfuji · 2024-11-08T19:48:07Z

@AlexanderRichert-NOAA Please make sure to push any changes here upstream afterwards. We want the packages to be 100% identical, if possible. I will create a PR for a3dd052 (cprnc) shortly.

climbfuji · 2024-11-08T20:05:04Z

spack PR for the cprnc bug fix: spack#47505

AlexanderRichert-NOAA · 2024-11-08T22:00:34Z

@climbfuji yep will do, I'm keeping a log, and I'll also provide a summary here when I'm done going through the list of conflicts.

climbfuji · 2024-11-08T22:09:53Z

Perfect, thanks a lot

…

________________________________ From: Alex Richert ***@***.***> Sent: Friday, November 8, 2024 3:00 PM To: JCSDA/spack ***@***.***> Cc: Dom Heinzeller ***@***.***>; Mention ***@***.***> Subject: Re: [JCSDA/spack] Update spack-stack-dev from spack develop as of 2024/10/31 (PR #482) @climbfuji<https://github.com/climbfuji> yep will do, I'm keeping a log, and I'll also provide a summary here when I'm done going through the list of conflicts. — Reply to this email directly, view it on GitHub<#482 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AB5C2RKBODF3DS7REYMR33TZ7UYBRAVCNFSM6AAAAABQ7LW4EWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRVHAYTMNRWG4>. You are receiving this because you were mentioned.Message ID: ***@***.***>

AlexanderRichert-NOAA · 2024-11-08T22:31:05Z

Here's my summary of the packages that were manually fixed for conflicts:
g2, gmake, harfbuzz, jasper, landsfcutil, openmpi, pixman, py-cffi, py-metpy, qt, sfcio, sigio, ufs-weather-model: good (in most cases no difference with main spack)
fontconfig: updated static deps logic
gdal: updated static deps logic for gdal, libgeotiff
libxkbcommon: updated static deps logic
py-setuptools-rust: removed duplicate py-typing-extensions dep

climbfuji · 2024-11-11T12:53:28Z

var/spack/repos/builtin/packages/libxkbcommon/package.py

@@ -1,4 +1,4 @@
-# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
+    # Copyright 2013-2024 Lawrence Livermore National Security, LLC and other


This should be reverted, or?

climbfuji · 2024-11-11T12:54:07Z

spack PR for the cprnc bug fix: spack#47505

The upstream PR for the cprnc bug fix was merged

wdconinc and others added 30 commits October 16, 2024 18:44

cyrus-sasl: patch v2.1.27:2.1.28 for gcc-14 (spack#47019)

62fd5d1

openldap: add v2.6.8; conflict gcc@14: for older (spack#47024)

5d2c67e

updated package specs for rust; removed redundant dependency due to i…

7bddcd2

…nherited cmake (spack#47022)

dakota: add conflicts and additional flags (spack#42906)

b661acf

Co-authored-by: Gava, Francesco <[email protected]> Co-authored-by: Bernhard Kaindl <[email protected]>

e4s external rocm ci: upgrade to v6.2.1 (spack#46871)

e4c2337

* e4s external rocm ci: upgrade to v6.2.1 * use ghcr.io/spack/spack/ubuntu22.04-runner-amd64-gcc-11.4-rocm6.2.1:2024.10.08 * magma +rocm: add entry for v6.2.1

isoquant: new package @3.6.1 (spack#47013)

178a8bb

Co-authored-by: LMS Bioinformatics <[email protected]>

caliper: add +python variant with pybind11 bindings (spack#47031)

1eb2cb9

* Updates Caliper recipe to build the new Python bindings * Implements setup_run_environment for Caliper to update PYTHONPATH

curl: mbedtls 3.6.0 bound should be forward not backward compat (spac…

aa6caf9

…k#47029) and add another backward compat bound for just 8.8

freetype: prefer 2.13.2 due to interface change in 2.13.3 (spack#47021)

5f56eee

clingo: use CMAKE_OSX_DEPLOYMENT_TARGET instead of *flags (spack#47043)

adaa0a4

llvm-amdgpu and composable-kernel: fix build failures (spack#46891)

962262a

e4s ci stacks: add fftx cpu, cuda, and rocm builds (spack#47004)

7ecdc17

* e4s ci stacks: add fftx cpu, cuda, and rocm builds * disable fftx+rocm due to spack github issue spack#47034 * e4s oneapi: fftx has spack build error spack#47048

unrar: add v7.0.9 (spack#47036)

0a7533a

otf-cpt: add new package for OTF-CPT (spack#47042)

75c71f7

Co-authored-by: Bernhard Kaindl <[email protected]>

voropp: swich to cmake (spack#47039)

049ade0

* voropp: migrate to cmake * lammps: update voropp dep

AOCL: add v5.0 (spack#46964)

2da812c

Adds variant to toggle use of rdpmc due to icl-utk-edu/papi#238 (spac…

9ba7af4

…k#47023)

curl: add v8.10.1 (fix CVE) (spack#46960)

580cc3c

pika: Add minimum CMake version requirement when using CUDA and C++20 (…

49a8e84

…spack#47077)

vtk-m: Add sycl option to vtk-m package (spack#46996)

b9e0914

Some unused methods in VTK-m resulted in compile errors. These were not discovered because many compilers ignore unused methods in templated classes, but the SYCL compiler for Aurora gave an error.

Added heyoka versions 6.0.0 & 6.1.0 (spack#47074)

4432f5a

fix use of traceback.format_exception (spack#47080)

c6bfe7c

Co-authored-by: Peter Scheibel <[email protected]>

py-torch: add v2.5.0 (spack#47069)

d8d41e9

implement prefix property for OneAPI compiler (spack#47066)

c006cb5

py-open-clip-torch: new package (spack#47049)

88c193b

Co-authored-by: Alex C Leute <[email protected]>

py-coca-pytorch: new package (spack#47051)

cfae194

* py-coca-pytorch: new package * [py-coca-pytorch] black --------- Co-authored-by: Alex C Leute <[email protected]>

py-ema-pytorch: new package (spack#47052)

53385f1

Co-authored-by: Alex C Leute <[email protected]>

py-x-clip: new package (spack#47060)

b2e28a0

Co-authored-by: Alex C Leute <[email protected]>

climbfuji added 2 commits October 31, 2024 18:43

Fix style errors in var/spack/repos/builtin/packages/fontconfig/packa…

57a28cf

…ge.py, var/spack/repos/builtin/packages/libxkbcommon/package.py, var/spack/repos/builtin/packages/openmpi/package.py

Attempt to fix failing Windows unit tests in Github actions

fee35e5

srherbener mentioned this pull request Nov 1, 2024

openmpi: add two_level_namespace variant for MacOS spack/spack#47202

Merged

climbfuji added 3 commits November 1, 2024 13:03

Fix merge conflict in lib/spack/spack/compilers/oneapi.py

b6b42d7

Fix merge conflict in .github/workflows/unit_tests.yaml

75c2f0f

Disable windows workflow for repos other than spack/spack

2f31a16

climbfuji changed the title ~~WIP Feature/update from spack dev 20241031~~ Update spack-stack-dev from spack develop as of 2024/10/31 Nov 4, 2024

climbfuji mentioned this pull request Nov 4, 2024

Update spack-stack-dev from spack develop as of 2024/10/31 JCSDA/spack-stack#1363

Merged

11 tasks

Bug fix for diab-disable flags in lib/spack/spack/compilers/intel.py …

7c25b42

…and lib/spack/spack/compilers/oneapi.py: set correct version ranges

climbfuji marked this pull request as ready for review November 5, 2024 19:35

climbfuji requested review from srherbener, RatkoVasic-NOAA and AlexanderRichert-NOAA November 5, 2024 19:35

srherbener approved these changes Nov 7, 2024

View reviewed changes

climbfuji and others added 4 commits November 8, 2024 08:00

Bug fix for package cprnc: set install rpath for cprnc executable

a3dd052

fontconfig: remove unneeded (commented) deps

b56af54

gdal: clean up static dep logic (use pkg-config)

590cfd7

libgeotiff: clean up static dep logic (use pkg-config)

cb48871

AlexanderRichert-NOAA added 2 commits November 8, 2024 14:02

libxkbcommon: update static libxml2 logic

ee41d52

py-setuptools-rust: remove duplicate py-typing-extensions dep

7321d81

climbfuji commented Nov 11, 2024

View reviewed changes

climbfuji merged commit b6ef265 into JCSDA:spack-stack-dev Nov 13, 2024
8 checks passed

climbfuji deleted the feature/update_from_spack_dev_20241031 branch November 13, 2024 22:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update spack-stack-dev from spack develop as of 2024/10/31 #482

Update spack-stack-dev from spack develop as of 2024/10/31 #482

climbfuji commented Nov 1, 2024 •

edited

Loading

AlexanderRichert-NOAA commented Nov 7, 2024

srherbener left a comment

climbfuji commented Nov 8, 2024

climbfuji commented Nov 8, 2024

AlexanderRichert-NOAA commented Nov 8, 2024

climbfuji commented Nov 8, 2024 via email

AlexanderRichert-NOAA commented Nov 8, 2024 •

edited

Loading

climbfuji Nov 11, 2024

climbfuji commented Nov 11, 2024

		@@ -1,4 +1,4 @@
		# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
		# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other

Update spack-stack-dev from spack develop as of 2024/10/31 #482

Update spack-stack-dev from spack develop as of 2024/10/31 #482

Conversation

climbfuji commented Nov 1, 2024 • edited Loading

Description

Testing

AlexanderRichert-NOAA commented Nov 7, 2024

srherbener left a comment

Choose a reason for hiding this comment

climbfuji commented Nov 8, 2024

climbfuji commented Nov 8, 2024

AlexanderRichert-NOAA commented Nov 8, 2024

climbfuji commented Nov 8, 2024 via email

AlexanderRichert-NOAA commented Nov 8, 2024 • edited Loading

climbfuji Nov 11, 2024

Choose a reason for hiding this comment

climbfuji commented Nov 11, 2024

climbfuji commented Nov 1, 2024 •

edited

Loading

AlexanderRichert-NOAA commented Nov 8, 2024 •

edited

Loading